A novel Multi-Layer Attention Framework for visual description prediction using bidirectional LSTM

Abstract The massive influx of text, images, and videos to the internet has recently increased challenge computer vision-based tasks in big data. Integrating visual data with natural language generate video explanations been a for decades. However, recent experiments on image/video captioning that employ Long-Short-Term-Memory (LSTM) have piqued interest researchers studying its possible application captioning. proposed architecture combines bidirectional multilayer LSTM (BiLSTM) encoder unidirectional decoder. innovative also considers temporal relations when creating superior global representations. In contrast majority prior work, most relevant features are selected utilized specifically purposes. Existing methods utilize single-layer attention mechanism linking input phrase meaning. This approach employs LSTMs extract characteristics from movies, construct links between multi-modal (words material) representations, sentences rich semantic coherence. addition, we evaluated performance suggested system using benchmark dataset obtained results reveal relative state-of-the-art works METEOR promising BLEU score. terms quantitative performance, outperforms existing methodologies.